Use oxide-tokio-rt rather than #[tokio::main]
#80
Conversation
As described in [RFD 579], we are now recommending certain non-default Tokio runtime configurations in all software we deploy in production. The [`oxide-tokio-rt`] crate provides a way to easily get these common recommended configurations by using its functions to initialize the runtime rather than `#[tokio::main]` Presently, `oxide-tokio-rt` does the following: - Enables DTrace probes for Tokio runtime events using [`tokio-dtrace`] - Disables the [LIFO slot optimization][lifo], which was the root cause of omicron#8334 But, we anticipate that if there are additional Tokio configurations which we want to recommend in the future, we'll add them here as well. This commit updates `dendrite` to initialize the Tokio runtime using `oxide-tokio-rt`. I've added a Clippy config for warning on uses of `#[tokio::main]`, as well, to avoid forgetting to use `oxide-tokio-rt` if new binaries are added. All the configs we set currently in `oxide-tokio-rt` requires the `tokio_unstable` RUSTFLAGS cfg, but it looks like that's already being set in order to use `tokio-conssole`, so that shouldn't be a problem. [RFD 579]: https://rfd.shared.oxide.computer/rfd/0579 [`oxide-tokio-rt`]: https://github.com/oxidecomputer/oxide-tokio-rt [`tokio-dtrace`]: https://github.com/oxidecomputer/tokio-dtrace [lifo]: https://rfd.shared.oxide.computer/rfd/0579#_disabling_the_lifo_slot_optimization
|
For the most part One way in which we are possibly more complicated than other components is that we need to play nicely with the standard illumos multithreading mechanisms. The tofino SDE on which we are built uses threads and mutexes internally for all of its asynchrony. There are times we call into thread-land from tokio code, and there are times we use tokio channels from the SDE's thread-land to send notifications back to the main body of our code. All of that is just to say that this should really get a workout on real hardware before being integrated, even though it look syntactically straightforward. |
Yeah, I think it's definitely worth testing the change the same way you'd exercise any other change, but I think it's unlikely to break anything. Most of the risk associated with picking up this stuff in other repos has actually just been build/dev environment complexity due to requiring |
|
(Sorry for the CI churn --- I couldn't get Dendrite to build locally due to some kind of libclang symbol versioning mess, so I thought I'd just let CI do it... 😅) |
As described in RFD 579, we are now recommending certain non-default Tokio runtime configurations in all software we deploy in production. The
oxide-tokio-rtcrate provides a way to easily get these common recommended configurations by using its functions to initialize the runtime rather than#[tokio::main]Presently,
oxide-tokio-rtdoes the following:tokio-dtraceBut, we anticipate that if there are additional Tokio configurations which we want to recommend in the future, we'll add them here as well.
This commit updates
dendriteto initialize the Tokio runtime usingoxide-tokio-rt. I've added a Clippy config for warning on uses of#[tokio::main], as well, to avoid forgetting to useoxide-tokio-rtif new binaries are added. All the configs we set currently inoxide-tokio-rtrequires thetokio_unstableRUSTFLAGS cfg, but it looks like that's already being set in order to usetokio-conssole, so that shouldn't be a problem.